Add copilot-instructions.md file#78584
Conversation
Would it apply if we asked for Copilot's review here? 😄 (Maybe with some sample code changes if needed) I'm particularly interested in the point about "NEVER produce a review body". Will it abide that? Is it actually desirable? I mean, if a human would have reasons to leave a top-level review comment, why wouldn't the same apply to an AI agent? |
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
I don't think so, I think Copilot works off what is in trunk. From the docs here:
Hmm, I wonder if we could open a PR using this branch as the base branch? 🤔
In the Copilot reviews I've seen, the top-level review comment usually restates what the PR is doing rather than leaving a useful review comment. Maybe we could soften the rule to something like: "Only write a review body when the finding spans multiple files and can't be attached to a specific line."? |
|
I opened another branch using this PR as the base, at #78643, and requested a review from Copilot. The changes included a deliberately-flawed change to The suppression rules around style/edges/tests appear to be working:
(I'm not highly opinionated about any of these rules by the way, this was all based on the sample of Copilot reviews I looked at.) But it still left a top-level comment restating what the PR is doing 😅 I don't think this is the most important rule, but it's frustrating that it just ignored it. I think we'd be best tweaking it to attempt to stop it from restating what the PR is doing: "Keep any review body to a single sentence stating only what was reviewed, not a restatement of the diff. Put all findings in inline comments." What do you think? |
2 participants
What?
Adds a
.github/copilot-instructions.mdfile to guide GitHub Copilot's PR review behaviour repo-wide.The file is split into three short sections:
@wordpress/data,__()/_x()/_n(), theblock-editor→editor→edit-postpackage layering rule).Why?
Copilot's default PR review in this repo tends toward long summary restatement and style observations that lint already covers, while occasionally producing comments based on hallucinated or unverified WordPress core APIs.
I sampled 50 recent Copilot reviews in this repo to form these rules:
How?
GitHub Copilot reads
.github/copilot-instructions.mdand merges its contents into every PR review in the repo.If this works well and folks agree, we could extend this to include further instruction files for more specific things, like code changes in specific packages having an additional set of guidance. I'm not sure how far we need to take this for it to be as helpful as possible.
Testing Instructions
Let's see if this improves the quality of Copilot's reviews once it's merged.
Use of AI Tools
I used Claude Code to sample the recent Copilot reviews and draft the instructions.